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PREFACE 


This bibliography was compiled for the 2006 Summer Institute at The Centre for Literacy - 
Accountability and Public Trust: Restoring the Balance. The selections include lectures, 
research studies, policy papers and government documents that describe and analyze recent 
concepts of accountability in the context of government funding in the non-profit sector in 
several countries. They look at issues of assessment and accountability in the broad field of 
education and more narrowly in the fields of adult literacy and adult basic education. Each 
section has been arranged in chronological order to reflect the evolution of ideas over the past 
two decades. Although far from exhaustive, the selected entries offer a set of essential readings 
on the topic and a point of entry for further research. 

The references, from Canada and a number of other countries, including the United States, the 
United Kingdom, Australia, and New Zealand, show the experience of accountability in the 
different countries to be remarkably similar. Put together, the references and annotations also 
form a narrative. 

Research for this bibliography included thorough searches of: the National Adult Literacy 
Database (NALD); the National Institute for Literacy (NILL); the Literacy Information and 
Communication System (LINCS) database; National Center for the Study of Adult Learning and 
Literacy (NCSALL); the Education Resources Information Center (ERIC); The Centre for 
Literacy resource collection; and the web sites of numerous national/international agencies with 
an acknowledged stake in literacy and lifelong learning. Lurther recommendations were 
contributed by participants at the Summer Institute. 

Search words used across platforms included: accountability; assessment; adult education; adult 
literacy; non-profit; audit; high-stakes testing. 


ACKNOWLEDGMENTS 

This bibliography was prepared by Paul Beaulieu, librarian/researcher at The Centre for Literacy, 
and edited by fellow staff members. Most annotations were written by the researcher [PB]; a 
few are abridged or edited versions of existing abstracts. The source is identified in square 
brackets. 

Bunding for the research was provided by the National Office of Literacy and Learning (NOLL), 
Human Resources and Social Development Canada (HRSDC). A grant for the print version was 
provided by Canada Post. 
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REFERENCES 


Concepts of Accountability 


Power, M. (1999). The Audit Society: Rituals of Verification. Oxford, UK: Oxford University 
Press. 

This book attempts to account for the proliferation of audits since the early 1980s in the 
United Kingdom and North America. Originally understood as a verification of financial 
statements, audit now applies to medical audits, technology audits, value for money audits, 
environmental audits, quality audits, teaching audits, and more. The author argues that the 
rise of auditing has its roots in political demands for accountability and control. There is 
now a whole industry devoted to checking up on people. But the issues of trust that lead to all 
this auditing also apply to the auditing process. The author questions the manner in which 
audits produce assurance and accountability. Audits can end up imposing values on the 
audited organization that are in conflict with the organization’s own values or mandate. [PB] 


O’Neill, O. (Ed.). (2002). A Question of Trust: The BBC Reith Lectures 2002. Cambridge, 
UK: Cambridge University Press. 

This book, the text of O’Neill’s 2002 Reith Lecture, brings a philosopher’s perspective to the 
issue of declining trust in public institutions, and whether demand for greater accountability 
of these institutions counters or actually feeds into this trend. O’Neill argues that demands 
for accountability stem from a culture of suspicion that is disseminated through streams of 
media and exploited by various groups for political purposes. The public may demand or be 
seen to demand tighter controls on public institutions to ensure that they do what they 
supposedly cannot otherwise be trusted to do. But O’Neill argues that in many cases such 
controls are counter-productive: someone trying to serve the public must devote inordinate 
amounts of time and energy to “being accountable” - time and energy that thereby does not 
go into serving the public. [PB] 


Stein, J. (2002). The Cult of Efficiency. (CBC Massey Lectures Series) Toronto, ON: 
Anansi Press. 

In the 2001 Massey Lectures, the author critically examines the ideology whereby the good 
of public services such as health care and education comes to be defined mainly in terms of 
“efficiency”. She argues that while accountability for such public services is needed, 
accountability mechanisms too often measure effectiveness in narrow, purely quantitative 
ways, and ignores the ways in which efficiency is not necessarily synonymous with quality. 
The demand for efficiency can in fact be a cloak for political agendas. The author discusses 
how the public might hold governments and public service providers genuinely accountable, 
as well as what results they should account for, and how. [PB] 
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Stein, J. (2002). Bad Translation or Double Standard? Productivity and 

Accountability across the Private, Public and Voluntary Sectors. The Review of 
Economic Performance and Social Progress, Volume 2. 

Conventional wisdom in public policy dictates that public services should be delivered as 
efficiently as possible. No one would speak in favour of wasting money. However, in the 
name of maximizing productivity, the public and voluntary sectors have adopted, or been made 
to adopt, an understanding of productivity that is considerably narrower than that of the private 
sector. This narrow understanding fails to take account of the realities of service delivery in 
the public and voluntary sectors, thus undermining their effectiveness. When accountability is 
defined by narrow concepts of efficiency, the concept of accountability is itself narrowed and 
emptied of meaning. True accountability is complex and many-sided. [PB] 
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Accountability Frameworks: Government and Non-Profits 


Panel on Accountability and Governance in the Voluntary Sector. (1999). 

Building on Strength: Improving Governance and Accountability in Canada 's Voluntary 
Sector. Voluntary Sector Roundtable. Retrieved May 2006 from 
http://www.vsr-trsb.net/ . 

This is the final report of the Panel, established in October 1997 by the Voluntary Sector 
Roundtable to review current accountability practices within the voluntary sector, develop 
proposed guidelines and practices to promote accountability within the sector, and lead a 
broad consultation on these proposals. The report notes the challenges to non-profits posed 
by growing demands for accountability, and makes recommendations to promote effective 
governance and accountability within the sector. It outlines an approach that combines 
regulation and self-regulation, based in part on the idea that that the answer to gaps in 
accountability is better regulation, not necessarily more. The report also notes the need to 
build capacity in organizations to enable them to better meet the challenge of accountability. 
[PB] 


Kroeger, A. (2000). Commentary - The HRD Affair: Reflections on Accountability in 

Government. Ottawa, ON: Canadian Policy Research Networks. Retrieved June 2006 
from http://www.cprn.com/ . 

In a speech to the Canadian Club on December 12, 2000, the author argues that the HRD 
scandal was blown out of proportion and contributed to an undesirable tightening of controls 
over government grants and contributions to organizations. He recounts how, in March 1998, 
the Human Resources Department (HRD, the precursor to HRSDC) initiated an audit of 
selected grants and contributions because of concern about the effects of cuts in the mid- 
1990’s on their ability to manage these programs. This audit turned up numerous examples of 
lax project monitoring by officials. Kroeger notes that while inadequate documentation and 
loose financial controls increase the l ik elihood of abuse, it does not necessarily mean that 
widespread abuse has in fact taken place. In this case, there was little evidence of actual 
wrongdoing by Department officials or by the organizations receiving funds. While officials 
may have made mistakes of discretion and flexibility, it is lamentable that the reaction to this 
shortcoming was a return to rigid bureaucratization. [PB] 


Canada. Voluntary Sector Task Force. (2001). An Accord Between the Government of 
Canada and the Voluntary Sector. Ottawa, ON: Voluntary Sector Initiative Secretariat. 
Retrieved May 2006 from http ;//www. vsi-isbc.ca/ . 

An accord between the Government of Canada and representatives of the voluntary sector 
meant to establish a basis of understanding between non-profit organizations and the 
Government of Canada in their mutual dealings, this document was written by a working 
group (the Joint Accord Table) of representatives from the Government of Canada and the 
voluntary sector. They heard from Canadians during consultations across the country in the 
summer and fall of 2001. Both sides make commitments. Government commitments include: 
To recognize and consider the implications of its legislation, regulations, policies and 
programs on voluntary sector organizations; to recognize its need to engage the voluntary 
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sector in open, informed and sustained dialogue in order that the sector may contribute its 
experience, expertise, knowledge, and ideas in developing better public policies and in the 
design and delivery of programs; and to address the issue of ministerial responsibility for the 
continued development of the relationship with the voluntary sector. [PB] 


Phillips, S. and Levasseur, K. (2004). The snakes and ladders of accountability: 

Contradictions between contracting and collaboration for Canada’s voluntary sector. 
Canadian Public Administration , v. 47, no. 4 (Winter 2004). Toronto ON: Institute of 
Public Administration of Canada. 

Governing in Canada is in transition between administrative models - from "new public 
management", with its contracting culture and associated accountability regime, towards a 
model of horizontal “governance” that emphasizes collaboration with a variety of non- 
governmental actors. The current accountability frameworks which are part and parcel of 
“new public management”, were given a new impetus by the HRD scandal of 2000. This 
article finds that their impact on voluntary organizations has been significant and 
overwhelmingly negative, imposing direct financial costs on voluntary organizations and 
stifling innovation. However, there is also a trend toward collaborative "governance" with 
non-profits which contradicts the controlling impulses embodied in current accountability 
frameworks. Looking at the Accord Between the Government of Canada and the Voluntary 
Sector and its Code of Good Practice on Funding as examples of this collaborative 
philosophy, the authors consider whether they might mitigate the negative effects of the 
accountability measures. [Publication Abstract] 


Abelson, J. and Gauvin, F-P- in collaboration with Mary Pat MacKinnon and Judy 
Watling. (2004). Transparency , Trust and Citizen Engagement: What Canadians 
Are Saying about Accountability. CPRN Research Report P-05. Ottawa, ON: Canadian 
Public Research Networks. Retrieved August 2006 from http ://www.cp rn.org/ . 

Based on public dialogues on the question of accountability, this report offers insight into 
public attitudes behind the tightening of accountability mechanisms for public funding of 
programs and services. Among other things, it reports widespread popular distrust of 
government management of their tax dollars, and demand for greater government 
accountability. Many participants in the dialogues said that they would be willing to pay 
more taxes for public services, on condition that there were clear mechanisms in place to 
ensure that money actually improved the services in question. [PB] 


Capgemini. (2004). Human Resources and Skills Development Canada (HRSDC) Final 
Report. Program Management Review : Grants & Contributions . Ottawa, ON: 
Government of Canada. Retrieved June 2006 from http://www.sdc.gc.ca/. 

This is the final report stemming from a review of HRSDC programs in the wake of a public 
scandal about alleged mismanagement. The report highlights a number of problems with the 
administration of HRSDC programs, among them levels of financial control disproportionate 
to the level of risk which caused department employees to focus on monitoring finances, and 
to pay insufficient attention to project results. [PB] 
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Aucoin, P. and Jarvis, M.D. (2005). Modernizing Government Accountability. 

Ottawa, ON.: Canadian School of Public Service. Retrieved June 2006 from 
http://www.myschool-monecole.gc.ca/ . 

This addresses the issue of political accountability - how government and people 
within it can be held accountable. The authors note that within the Canadian government 
virtually all those to whom an account is due are themselves accountable to someone higher 
up in the hierarchy, so that there can be said to be a chain of political accountability. This 
"chain" is not straightforward, however, given the complexities of our parliamentary and 
bureaucratic system. The authors look into the current mechanisms of political accountability 
and their limitations, and compare them to other possible models. [PB] 


Clark, I.D. and Swain, H. (2005). Distinguishing the real from the surreal in 

management reform: suggestions for beleaguered administrators in the government of 
Canada. Canadian Public Administration / Administration Publique du Canada Volume 
48, No. 4 ( Winter/Hiver 2005), pp. 453-476. Toronto, ON: Institute of Public 
Administration of Canada. 

The authors illustrate the practical limitations of current management improvement 
initiatives in the Government of Canada, suggesting that the most noteworthy is their failure 
to recognize limitations. These initiatives, which include accountability mechanisms such as 
performance measurement and performance audit, are based on utopian frameworks. 
Managers subjected to their surreal demands are faced with an ethical dilemma: as public 
servants, they have a duty to comply with the requirements, yet meeting them conflicts with 
their duty to manage people and resources sensibly. The authors offer advice to managers on 
ways to improve management in departments despite the surreal demands. [PB] 


Canada. House of Commons. (2005). An Examination of New Directives Governing 
Contribution Agreements for Selected Programs Delivered on Behalf of Human 
Resources and Skills Development Canada: Report of the Standing Committee on 
Human Resources, Skills Development, Social Development and Status for Persons with 
Disabilities. Retrieved June 2006 from http://cmte.parl.gc.ca/cmte/ . 

This is a report on the problems in implementing new policy directives designed to improve 
the administration, management and accountability of contribution programs funded 
through Human Resources and Skills Development Canada (HRSDC). These directives were 
a response to the “Billion Dollar Boondoggle” scandal that arose from an internal audit of 
selected HRSDC programs carried out in 1999-2000. Problems included a failure by the 
Department to consult with groups during the development and implementation of the 
directives, which ran counter to the commitment outlined in An Accord Between the 
Government of Canada and the Voluntary Sector. The report cites problems with the 
Department’s new call-for-proposals process, and notes complaints about “forensic” 
accounting and micro-management in the negotiation and management of contribution 
agreements, as well as over-emphasis on financial inputs and insufficient attention to 
program effectiveness and outcomes. [PB] 
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Imagine Canada; Canadian Council on Social Development; Canadian Policy 

Research Networks (2005) Submission to the Commission of Inquiry into the Sponsorship 
Program and Advertising Activities. Retrieved May 2006 from http ://www.voluntary- 
s.ca/eng/publications/index.cfm . 

A submission made on behalf of Canadian non-profit organizations to the Gomery 
Commission investigating the sponsorship scandal, this brief expresses concern that, in a bid 
to fix the problems revealed in that scandal, already onerous reporting requirements would be 
made even more burdensome. It outlines some of the major complaints from the non-profit 
sector concerning the current public funding regime and accountability frameworks. [PB] 


Levasseur, K. and Phillips, S.D. (2005). Square Pegs In Round Holes: Vertical and 
Horizontal Accountability in Voluntary Sector Contracting. The Philanthropist, v. 19, 
no. 3 (July 2005). Toronto, ON: The Agora Foundation. 

This article examines how project funding by the Government of Canada demands 
accountability of voluntary sector organizations to government at the expense of 
accountability to partners, users, and stakeholders. It describes this relationship of 
accountability between government and voluntary sector organisations as top-down, or 
“vertical”. It argues that the effects of vertical accountability, with its emphasis on financial 
reporting, does little to enhance accountability for results, and that project funding is not only 
unable to nurture meaningful accountability, but that it also has a detrimental impact on 
voluntary organizations. It maintains that a new, less restrictive, funding tool is needed if the 
federal government is to contribute to capacity building and innovation in the voluntary 
sector, and to build a more constructive, collaborative relationship with it over the long run. 

It concludes by discussing how such a tool might be designed to nurture more meaningful 
accountability, both vertically and horizontally. [Publication Abstract] 


Lewis (2005). Public Institution? or Public Nuisance? HRSDC, the community 

sector's increasingly dysfunctional 'partner'. Making Waves 15 (4): 2-6. Vernon, BC: 
Centre for Community Enterprise. Retrieved October 2006 at 
http://www.cedworks.com/hrsdc.html . 

The author claims that HRSDC rules meant to set rigorous standards for accountability are 
sapping the vitality from once vibrant programs. The article focuses mainly on employment 
support initiatives and social enterprises. The author argues that "hyper-categorization" of 
programs and "twisted micro-management" promote waste, inefficiency and chaos by 
interfering with overall coherence of programs. According to people in the field, rules punish 
entrepreneurial behaviour in the community sector. HRSDC employees are themselves 
constrained by these rules and by a siege mentality under which the top priority is not getting 
embroiled in any further scandals. [PB] 
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MacNeil, R. (2005). Accountability and Community Economic Development: The 

Funder-Governed NGO. Toronto, ON: Canadian Research Alliance for Community 
Innovation and Networking. Retrieved June 2006 from www3.fis.utoronto.ca 

Part literature review, part case study, this article examines the question of how non-profit 
organizations can reconcile their responsibilities to the community with demands for 
accountability from funders which can cause projects to reflect funders’ priorities. MacNeil 
notes the frustration of people in the community sector over new accountability rules. He 
uses a case study to illustrate problems, and explores the context in which these problems are 
occurring - a government caught between conflicting management models and reacting to the 
HRD scandal of 2000. [PB] 
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Assessment and Accountability in Education 


Kirst, M. (1990). Accountability: Implications for State and Local Policymakers. {Policy 
Perspectives) Washington, DC: Superintendent of Documents, U.S. Government 
Printing Office. 

This is a guide to the issues in designing and implementing accountability models for U.S. 
public schools, and discusses different concepts of accountability, their history in education, 
and different approaches to accountability. It notes the ongoing conflict between the need to 
respect the professionalism of educators and the need for those receiving public money to 
account for its use. It identifies six approaches to accountability: 1) through performance 
reporting; (2) through monitoring and compliance with standards/regulations, (3) through 
incentive systems; (4) through reliance on the market; (5) through changing the locus of 
authority or control of schools; and (6) through changing professional roles. It suggests that 
policymakers should combine two or more of these approaches for best results. [PB] 


Leithwood, K.; Edge, K.; and Jantzi, D. (1999). Educational Accountability: The State of the 
Art. Gutersloh, Germany. Bertelsmann Foundation Publishers. 

Government policy on education has come to emphasize mechanisms of accountability as a 
means of ensuring and improving educational outcomes. The book attempts to respond to the 
following five issues: Who is accountable, to whom, for what, at what level, and with what 
consequences? It describes a framework developed by the Ontario Institute for Studies in 
Education for understanding educational accountability in response to these issues, which 
identifies four approaches to accountability: the market competition approach, the 
decentralized decision-making approach, the professional, and the management approach. 

The book describes how member countries of the International Network of Innovative School 
Systems (Canada, Germany, Hungary, the Netherlands, New Zealand, Norway, Scotland, and 
Switzerland) are using and combining these approaches in designing and implementing their 
accountability tools, and considers implications for future policy and practice. [PB] 


Reeves, D. (2003). Accountability for learning: how teachers and school leaders can 
take charge. Alexandria, VA: Association for Supervision and Curriculum 
Development. 

Accountability has become a “dirty word” for many educators, and in most school systems, 
has become synonymous with test scores. The author argues that teachers have to take the 
initiative if they want accountability mechanisms to move beyond narrow accounting 
exercises to provide constructive and useful feedback for them and their students. He 
encourages them to take the lead in developing “holistic” student-centred accountability 
systems that will tell the stories behind the numbers. Such systems would demonstrate the 
different variables that produce educational outcomes, providing a clearer picture of the 
reasons for success or failure, and a clearer route to improvement. He shows how such 
accountability systems can enhance teacher motivation and lead to significant improvements 
in student achievement and equity, even in traditionally low-performing schools. [PB] 
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Sirotnik, K.A. (Ed.) (2004). Holding Accountability Accountable: What Ought to 
Matter in Public Education. New York, NY: Teachers College Press. 

This essay collection critiques current high-stakes accountability systems that have taken 
hold in the U.S. public education system. The editor asserts that accountability practices in 
public education are “miseducative”, “misdirected and misanthropic”, grounded in the 
ideology of “accountablism” which couches all educational objectives in behavioural terms 
and uses coercion based on the consequences of narrowly defined outcomes to control 
behaviour. The articles illustrate the negative effects of heavy-handed accountability systems 
on educators and students. [PB] 


Wilson, M. (Ed). (2004). Towards Coherence between Classroom Assessment and 

Accountability. 103rd Yearbook of the National Society for the Study of Education. 
Chicago: University of Chicago Press. 

This volume seeks to restore the classroom to its rightful place at the centre of analyses of 
accountability through assessment. Ideally, assessment should encourage a two-way flow of 
information between the classroom and systems outside, but "standards-based" frameworks 
tend to direct the flow of accountability from the outside into the classroom. Yet standardized 
tests sent into the classroom to produce quantifiable “outcomes” represent only a fraction of 
all the assessment that goes on there - most of it generated within the classroom by the 
teacher. The articles look at different ways in which this rich source of assessment could be 
tapped to create a more complete picture of the learning that is occurring. Five source 
chapters describe successful classroom assessment models developed in partnership with 
teachers. The articles offer a range of perspectives on the issues of classroom assessment, 
standardized testing, and accountability. [PB] 
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Assessment and Accountability in Adult Education 


Armstrong P. (1986). “Called into account: evaluation, accountability and objectivity” in 
M. Zukas (ed), Papers from the Sixteenth Annual Conference of the Standing 
Conference on University Teaching and Research in the Education of Adults. Leeds, UK: 
University of Leeds/SCUTREA, 1986 pp 134-141. 

This paper examines the politics of evaluation research, using a case study to illustrate the 
problems such politics pose in relation to accountability. Processes of evaluation to call 
someone to account are looked at as relations of power. The author notes that such a process 
cannot be objective because of this power dynamic, and because the values of the party 
seeking the account will determine what is measured in the assessment. He calls on 
researchers to be aware of the political nature of the research they may be called upon to do. 
[PB] 


Ziegler, S. (1996). The Effectiveness of Adult Literacy Education: A Review of Issues 

and Outcomes-based Evaluation of Literacy Programs. Toronto, ON: Ontario Literacy 
Coalition. 

This report examines issues associated with demonstrating the effectiveness of adult literacy 
programs and reviews research. The report is divided into three sections. Section One notes 
that there are few empirical studies that show the impacts of literacy programs or identify 
predictors of program success - among the reasons for this is a lack of common criteria for 
evaluating performance. However, reliable data on outcomes would benefit practitioners, 
students and policymakers. Section Two reviews research on variables affecting the 
outcomes of literacy programs. Section Three provides conclusions and recommendations, 
and poses questions for further research. [PB] 


Vella, J.; Berardinelli, P.; Burrow, J. (1997). How Do They Know They Know: 

Evaluating Adult Learning. San Francisco, CA: Jossey-Bass. 

This book takes a popular education approach to the issue of assessment for accountability. It 
describes how the Jubilee Popular Education Center developed an "accountability process”, 
starting with a clear identification of the purpose of, or results expected from, the education 
program. These include learning, transfer (learning applied by learners in their work), and 
impact (improvement in the learners' organisation as a result of transfer). The authors take 
the reader step-by-step through their approach, which integrates evaluation into program 
planning, engaging learners as partners throughout. The concepts are related to real-life 
situations, demonstrating how they can apply to a variety of settings. [PB] 
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Alamprese, J. (1998) Promoting Systemic Change in Adult Education and 

Employment and Training. Columbus, OH: Center for Education and Training for 
Employment. Available at http ://www-tcall.tamu.edu/ 

This paper describes approaches to systemic reform in U.S. adult education programs, based 
on the author’s work in documenting state program improvement initiatives in California, 
Connecticut, Pennsylvania, and other states. It begins with a definition of systemic change as 
well as a framework for understanding it. It presents approaches to systemic reform as they 
relate to four possible areas that policymakers can use: (1) stimulating the use of innovative 
management, assessment, and instructional practices; (2) developing policy to support 
change strategies; (3) facilitating organizational and professional learning; and (4) co- 
ordinating and collaborating within and across states to leverage support for adult education. 
In describing these approaches, Alamprese examines factors to consider in the development 
of accountability systems for adult and vocational education and employment and training 
systems, and looks at ways in which data can be used in managing and improving programs. 
[ERIC + PB] 


Merrifield, J. (1998). Contested Ground: Performance Accountability in Adult Basic 

Education. NCSALL Reports #1. Retrieved June 16, 2004 from the National Center for 
the Study of Adult Learning and Literacy web site: http ://www.ncsall.net/ . 

Practitioners of adult basic education (ABE) are being challenged to demonstrate its 
effectiveness in terms of producing student and societal outcomes. This policy paper draws 
on the literature from education, government and management and other fields to lay out key 
issues in performance accountability and present recommendations for policy and action. 

One key issue is whether all those with an interest in ABE can agree on what constitutes 
authentic indicators of program effectiveness. The question of what counts as good 
performance comes down to what people value. To avoid being experienced as the 
imposition of some people’s values on others, accountability relationships should be mutual 
among different interested parties. Such a “web” of accountability relationships would 
accommodate the values of all those involved in the enterprise of ABE. Merrifield also 
recommends that multiple assessment tools be developed and used to reflect the complex 
reality of ABE. [PB] 

See also- Demetrion, G. (2000). Reflecting on Culture Wars in Adult Literacy Education: 

Exploring Critical Issues in "Contested Ground"(p. 19) 
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Garner, B. (1999). Nationwide Accountability: the National Reporting System. 

Focus on Basics: Connecting Research & Practice, Vol. 3, Issue B. Retrieved June 20, 
2006 from the NCSALL database at http://www.ncsall.net/ . 

Under growing pressure to document that participation in literacy programs leads to positive 
results, federal and state administrators of the U.S. adult basic education system embarked on 
an effort to improve the national data reporting system for this education system. By 
redesigning the National Reporting System (NRS), they also hoped to link it more closely to 
program accountability and improvement. A pilot test of the new system found difficulties in 
obtaining data, especially follow-up data from former students. [PB] 


Goethel, J. and Gabler, C. (1999). Accountability in a Multi-Faceted Program. 

Focus on Basics: Connecting Research & Practice, Vol. 3, Issue B. Retrieved June 20, 
2006 from the NCSALL database at http://www.ncsall.net/ . 

Increasing partnerships means satisfying more partners. This article shows how one volunteer 
literacy program (Literacy Volunteers of America - Chippewa Valley) has been clarifying 
goals and improving accountability systems to deal with the challenges of accountability as it 
becomes larger and more complex. The managers of this program describe themselves as 
being accountable to their students, their tutors, their staff, and to fourteen different funders. 
Their organization has created a strategic plan encompassing its myriad activities, which has 
enabled it to clearly identify goals and target improvements. In using it, programs are now 
being assessed according to clear criteria. [PB] 


Keenan, C. (1999). Guiding Improvement: Pennsylvania's Odyssey. Focus on 

Basics: Connecting Research & Practice, Vol. 3, Issue B, June 1999. Retrieved June 20, 
2006 from the NCSALL database at http://www.ncsall.net/ . 

This article describes the creation of an accountability system in Pennsylvania. Combining 
direction from the top and innovation at the local level, the state aimed to achieve a situation 
of mutual accountability. The state would be more responsive to local needs, ease access to 
funding and invest in teacher training and experimental programs. Local programs would 
bring teachers, tutors, program coordinators and administrators together to solve problems 
identified in the assessment data. Drawing on experiences from pilot projects, they built on 
current local practices to develop state- wide standards of assessment. [PB] 


Sticht, T.G. (1999). Testing and Accountability in Adult Literacy Education. 

Retrieved June 21, 2006, from http://www.nald.ca/fulltext/sticht/testing/cover.htm . 

This report discusses the design and evaluation of workplace literacy programs and the use of 
standardized tests in program evaluation and accountability as well as the concepts behind 
the use of these tests. The author notes that while federal funders want "objective" and 
"measurable" outcomes from local workplace literacy programs, the major concern of local 
programs is to meet the needs of their adult learners and partners. Meanwhile, assessment 
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tools such as standardized tests are often inappropriate, as they do not measure how well the 
programs enable learners to meet the literacy requirements of actual jobs. [PB] 


Zepke, K. (1999). Autonomy, Accountability in Adult Education and the Politics of Official 
Knowledge. New Zealand Journal of Adult Learning. 27 (2) pp. 31-44. 

Although the nature of knowledge is contested, "official knowledge" is promulgated in 
national standards such as New Zealand's National Qualifications Framework. Autonomy 
and accountability are used in the politics of official knowledge by competing interests. 
Critical theory provides guidelines for adult educators to deal with the politics. [ERIC] 


Battell, E. (2001). Naming the Magic: Non-Academic Outcomes in Basic 
Literacy. Province of British Columbia. Retrieved June 21, 2006, from 
http://www.nald.ca/fulltext/magic/cover.htm 

This report presents a description of the following techniques for documenting non-academic 
outcomes (NAOs) for literacy students: A Multi-Media Approach, Using Journals, Goal 
Setting by Learners, Anecdotal Reporting and End-of-Term Reports (combination of two 
techniques) and Questions for Developing Awareness. It documents the difficulties 
encountered during the field-testing, outlines what was learned during the field-testing 
process and makes six recommendations, based on the project, that call for more extensive 
and focused study of non-academic outcomes. [From NALD] 

Demetrion, G. (2000). Reflecting on Culture Wars in Adult Literacy Education: 

Exploring Critical Issues in "Contested Ground". Retrieved June 21, 2006, from 
http://www.nald.ca/fulltext/cultrwar/cover.htm 

While the theories of New Literacy Studies are being applied in teaching, they have had 
much less currency at the level of educational systems and policies - institutions, funding, 
and accountability. A shift in the understanding of literacy means that performance is 
defined differently and requires a different approach to accountability. Perspective on what 
is good performance in adult education, and what should be measured, depends on context 
and position. Learners may want a program that treats them with respect, allows them to feel 
successful, provides them with the learning opportunities they want, and supports results that 
are important to them. Policy-makers may simply want a program that gets people into jobs. 
However, as Juliet Merrifield (see p. 16) maintains, the concept of functional literacy should 
be laid to rest. The concept is flawed, its definition arbitrary, its measurement problematic. 
The phenomenon of "functioning in life" cannot readily be equated with literacy. Adults with 
limited literacy skills should be credited for the skills and knowledge they have. Educators 
should build on and extend this knowledge and skill, based on the needs, desires, and 
interests of the adult learners, rather than dwell on measuring how "functional" a learner is or 
needs to become, according to standardized tests. [From NALD] 
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Hodkinson, P. and Bloomer, M. (2000). Accountability, Audit and Exclusion in 

Further and Higher Education, paper presented at SCUTREA Annual Conference, 
University of Nottingham, 3-5 July. 

This paper presents research evidence about the complex causes of drop-out from further 
education, and questions the pertinence of course completion as a measure of the efficacy of 
a further education program. The authors note that many of the reasons for dropping out are 
beyond the institutions’ control, and do not adequately measure what learning has actually 
gone on. Furthermore, giving such prominence to retention outcome measures is a 
distraction from seriously working to widen participation in lifelong learning. 

[Author Abstract] 


Cora, M. (2001). Layers, Brushes, and Multi-Lane Highways: Examining Accountability 
in a Non-Traditional Program. Adventures in Assessment, Volume 13, pp.9-12. 

Retrieved July 2006 from http://www.sabes.org/resources/adventures/ 

Reflecting on her experiences as director of the Swearer Center for Public Service in 
Providence, Rhode Island, Cora argues that accountability should be seen as a "multi-lane 
highway", or "in layers". There are many relationships involved in adult education that 
involve some form of accountability. While the institution is accountable to the funder, its 
accountability to the adult learners is more fundamental. Working backwards from this 
primal relation of accountability, other relations emerge: To meet the needs of the adult 
learners, the institution must be accountable to the tutors to support their teaching; the tutors 
must be accountable for their performance, and so on. It is important that all players identify 
and articulate their expectations of each other to create clear channels of accountability. [PB] 


Kelly, J. (2001). The Adventure Continues. Adventures in Assessment, Volume 

13, pp. 13-16 Retrieved July 2006 from http://www.sabes.org/resources/adventures/ 

This article describes how assessment in adult basic education programs has changed over 
the previous ten years. Among the changes is the degree of accountability required by most 
funding sources. Funders now want to know much more about many more things. Since each 
may require that program efficacy be demonstrated in a different way, programs with 
multiple funders can become entangled in multiple reporting requirements. While being 
accountable is reasonable, it is unreasonable to have to devote more hours to fulfilling the 
funders' ideas of accountability than to doing useful assessment with learners. [PB] 


Gabb, S. (2001). Authentic Goal Setting with ABE Learners: Accountability for 

Programs or Process for Learning? Adventures in Assessment, Volume 13, pp. 17-23. 
Retrieved July 2006 from http://www.sabes.org/resources/adventures/ 

The National Reporting System (NRS) of the United States has as its stated goal “to 
establish an accountability system for the federally-funded adult basic education programs.” 
The official Implementation Guidelines require learner goal setting as a measure of program 
performance. Meanwhile, practitioners across the state of Massachusetts have encouraged 
recognition of learner-generated goals. However, the purposes for goal setting envisioned by 
practitioners and the process for supporting learners in setting goals, differ radically from the 
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procedures outlined in the NRS Measures and Methods. The article explores the implications 
of this tension and ways of dealing with it. [PB] 


Reuys, S. (2001). New Accountability Rules Pose Dilemma for Programs. 

Adventures in Assessment, Volume 13, pp. 5-8. Retrieved July 2006 from 
http://www.sabes.org/resources/adventures/ 

This article looks at one challenge posed to programs as they try to comply with the 
requirements of the new National Reporting System (NRS) and the state's SMARTT data 
management system. It is how to assess and report individual student progress in literacy and 
language learning in a way that meets the new reporting requirements yet provides teachers 
and students with assessment information that is meaningful, accurate, and useful. The 
article looks at the following possible approaches to this issue: simply adopting standardized 
tests as the sole means of assessment; trying to use alternative assessment methods and 
“translate” the results thereby generated into “standardized” results (GLE's and SPL's); or 
combining standardized tests and alternative assessments. [PB] 


Project Literacy Victoria. (2001). Outcome Measurement for a Community Literacy 
Program. Retrieved June 21, 2006, from http://www.plv.ca/outcome/cover.htm 

In the non-profit sector there is increasing emphasis on accountability and a trend for funders 
to demand measurable outcomes. In 1999 forty basic literacy instructors in B.C. met to 
discuss how the non-academic outcomes in the lives of learners could be documented. They 
drafted six techniques, to be further developed by several conference participants in 
consultation with the project co-ordinator. The report describes these techniques and 
documents how they were field-tested by literacy practitioners including instructors, tutors, 
and tutor co-ordinators. It documents the difficulties encountered during the field-testing: 
timing, writing for instructors with a range of experience, and confusion about instructions. 

It also outlines what was learned about the techniques, the non-academic outcomes 
themselves, the field, the field-testing, and, importantly, self-reflection. Finally, it makes six 
recommendations that call for more extensive and focused study of non-academic outcomes, 
and for technical assistance to non-profits to adopt outcome measurement tools. [PB] 


Derrick, J. (2002). What Could a Socio-Cultural Approach to Literacy, Numeracy and 
ESOL Practice Be, and How Does This Align to Current Practice [Paper one for 
Economic and Social Research Council Seminar, 18 October]. ESRC Adult Basic 
Education Seminar Series. Retrieved June 2006 from 
http://www.education.ed.ac.uk/hce/ABE-seminars/ 

This paper discusses the significance of England’s national performance measurement 
systems to program-level practices in Literacy, Numeracy and ESOL (LNE), and assesses 
the extent to which the current adult education framework supports socio-cultural 
approaches to instruction and assessment. Reflects on the question of how England may 
establish a regulatory framework that ensures consistent, efficient and effective support for 
learning, without alienating the learning process from the social field. [CE] 
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Derrick, J. (2002). A Socio- Cultural Approach To Literacy Numeracy And 

ESOL Practice: A Practice Perspective. [Paper two for Economic and Social Research 
Council Seminar , October 18]. ESRC Adult Basic Education Seminar Series. Retrieved 
June 2006 from: http://www.education.ed.ac.uk/hce/ABE-seminars/papers/ABEl- 
JayDerrick2.pdf 

This paper focuses on performance measurement systems for LNE (Literacy, Numeracy and 
ESOL) work, particularly in England. The author argues that the detachment of policy 
development on Literacy, Numeracy and ESOL (LNE) from adult education in general 
resulted in the loss of a socio-cultural perspective. A socio-cultural perspective is rooted in 
cultural and social practice, and promotes informal as well as formal learning. This is 
contrasted to a process in which learning is abstracted from the social and cultural realities 
of those involved. The English framework focuses on external assessment and national tests. 
The author suggests a need to develop performance indicators more congenial to a socio- 
cultural approach. [PB] 


Mislevy, R. J. and Knowles K.T., [Eds.] (2002). Performance Assessments for Adult 
Education: Exploring the Measurement Issues Report of a Workshop. 

Committee for the Workshop on Alternatives for Assessing Adult Education and 
Literacy Programs, Board on Testing and Assessment, Center for Education, Division 
of Behavioral and Social Sciences and Education, National Research Council. 
Washington, DC: National Academy Press 

This report considers the measurement issues of using performance assessment for 
accountability purposes, given that the Workforce Investment Act of 1998 requires states to 
establish comprehensive accountability systems for adult education programs, featuring the 
gathering of data on certain core measures. It includes a summary of options and strategies 
discussed for practical steps to make performance assessments in adult education useful to 
educators and students and adequate for national reporting purposes without requiring 
excessive amounts of staff time or money. [PB] 


Pheasey, A. (2002). What Do Adult Literacy Students Think Being Literate Is? Retrieved 
June 27, 2006, from http://www.nald.ca/ripal/Resourcs/publicat/whatdo/cover.htm 

As a member of the RiPAL Network, the author investigated what adult literacy students 
think being literate is. In this report, based on interviewing students at The Learning Centre 
in Edmonton, Alberta, Pheasey shares what she learned about the question “What do literacy 
students think being literate is?”. She also wanted to know what notions of literacy or 
personal goals learners had when they came into adult literacy programs. From a review of 
the literature, she found that the meaning of the term “literacy” is often unclear. Through 
taped individual and small group interviews she found that participants were motivated to 
come to the program by the need for social interaction and the need to acquire skills (reading 
and writing). Although participants did not associate being literate with economic outcomes, 
they did associate levels of literacy with social class, and some reported feelings of exclusion 
associated with their low levels of literacy. [PB] 


20 


Van Duzer, C. (2002). Issues in Accountability and Assessment for Adult ESL 
Instruction. National Center for ESL Literacy Education. Available online at 
http ://www.cal.org/caela/esl resources/ 

Throughout the 1990s, legislation in the U.S. increasingly required federally funded 
programs to be more accountable. Lor adult education, these requirements have intensified 
the debate among practitioners, researchers, and policy makers over what constitutes success 
and how to measure it. At the same time, the number of English language learners enrolled 
in adult education programs has been growing, particularly in areas of the country that have 
not previously had many immigrants. New programs are being established to meet the 
demand for English as a Second Language (ESL) instruction, and existing programs are 
expanding. This Q&A describes the legislative background of current accountability 
requirements for ESL programs, the issues involved in testing level gain, and critical 
questions whose answers can lead the field forward. [NALD] 


Alderson, L. and Twiss, D. (2003). Literacy for Women on the Streets. Retrieved 
June 27, 2006, from http://www.nald.ca/fulltext/litforwm/cover.htm 

This is a report on a research project designed to examine the impact of literacy activities on 
the lives of women working as sex-trade workers in the Downtown Easts ide of Vancouver. 
An integral part of the literacy program, the project shows how different forms of 
documentation can be used to demonstrate the extent to which a project is achieving desired 
results. The authors describe the research and share the tale of their two-year experience and 
their findings to give funders, literacy and community development workers, and women 
living and working on the street, more confidence about embarking on literacy programs 
together. [From NALD] 


Belzer, A. (2003). Living With It: Federal Policy Implementation In Adult Basic 

Education. The Cases Of The Workforce Investment Act And Welfare Reform. NCSALL 
Reports No. 24. Cambridge, MA: National Center for the Study of Adult Learning and 
Literacy. Retrieved June 6, 2006 from the NCSALL database at http ://www.ncsall.net/ 

This study gathered data from six states through in depth interviews on practitioners' 
perceptions of how the Workforce Investment Act and welfare reform changed their 
practices. As the policy changes made their way from the federal level, states, programs and 
practitioners were able to respond to and reshape policy to their own contexts. However, 
while accountability requirements raised expectations for educators, there was not much 
additional support or resources to enable educators to better meet these expectations. 
Consequently, the requirements often simply created additional burdens on instructors. The 
author recommends that policymakers ensure that policy provides clear direction rather than 
mixed signals, and concern themselves with improving the professional climate for 
educators. [PB] 
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Crowther, J. (2003). Accounting and Accountability In Adult and Community Education: 
Reflections On The Scottish Experience. Speaking in Tongues: the Languages of 
Lifelong Learning, pp. 45-50, SCUTREA Conference Proceedings. 

This article addresses the changing language of adult and community education in Scotland 
through its incorporation into the "managerial politics" of government, and the implications 
of that change for practice. It traces a shift from a discourse informed by a humanist 
ideology stressing personal growth and empowerment to a discourse informed by neo-liberal 
ideology, in which international competitiveness requires the development of "human 
capital". Neo-liberal ideology views learners as being primarily consumers and sellers of 
labour who are responsible for increasing their value in the marketplace through lifelong 
learning, rather than relying on the state for help. It is in this context that the current drive for 
systems of evaluation and accountability needs to be understood. [PB] 

Eckert, E., & Bell. A. (2004). Authentic Accountability In Literacy Education. Adult 
Basic Education, 14 (3), 174-188. 

Educators are resentful at having accountability requirements imposed on them when results 
judged to be important by such systems often conflict with what is important to educators 
and learners. These systems rely on a positivist approach that assumes an objective reality 
composed of variables that can be isolated from other variables. The result is a “one-size - 
fits-all” approach to assessment. As an alternative, the authors propose a framework for 
“authentic accountability” that reflects the complexity of interactions between learners, 
teachers and programs, and allows the voices of learners to be heard. They suggest some 
ways to put this framework into practice. [PB] 


Herrington, M. (2004). The Both... and... And of Everything. Literacies, #3, 9- 
10 (Spring 2004) Available at http://www.literacviournal.ca/ 

This article addresses the day-to-day issues for literacy workers of trying to construct a 
professional space for development when quite different concepts of literacy coexist and 
compete around them. Many literacy practitioners have to find ways of working with 
opposing concepts at the same time. Recent policy initiatives in the UK - the Skills for Life 
initiative in particular - demonstrate how limited the policy frameworks tend to be compared 
with thinking in the field, [author + PB] 


Jackson, N. (2004). What Counts as Literacy Work? Literacies, #3, 9-10. 

Available online at http ://ww w.literac vi ournal.ca/ 

This article examines the issues surrounding the development of national literacy policy 
frameworks in Canada. In June 2003, a Parliamentary committee report called for a first-ever 
pan-Canadian accord on adult literacy and numeracy skills development that would commit 
different levels of government to work together to increase the proportion of Canadians with 
higher- level literacy skills. However, while the prospect of a national system has promise, 
the experience of other countries shows that it also has its share of pitfalls. Complaints about 
the distorting effects of the policy and reporting frameworks that are meant to ensure 
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accountability in such systems are remarkably similar across national boundaries. These 
frameworks are said to be imposed and not reflective of realities in the field. Literacy 
practitioners either deal with them by “producing the right numbers” by hook or by crook, or 
are worn down and demoralized by the demands placed on them. Jackson recommends that 
literacy practitioners become more aware of and active in the policy discussions that are 
affecting their work. [PB] 


Maclachlan, K. (2004). Changing Policies, Changing Practices, Changing Approaches to 
Adult Literacies Learning. Adult Education and Poverty Reduction: Proceedings of the 
International Conference, University of Botswana, July 2004 

Adult literacy education can achieve significant results if it is part of a broad, multi-faceted 
strategy for reducing poverty. For that to happen, however, such education must transcend 
the dominant understanding of literacies as a set of technical skills taught to overcome s ki ll 
deficits, and detached from their social context. It should be based on a relational concept of 
literacies in which learners build on what they already know to develop social 
communicative practices of engagement with the social contexts of their lives such as 
family, the workplace, and the community. In this respect, Scotland’s adult literacy and 
numeracy policy compares favourably with that of England, where programs must meet 
standardized national targets to receive funding. Scotland’s system enables learners to 
articulate their own goals for learning, and the success of their learning is measured against 
those goals. [PB] 


Sticht, T.G. (2004). The Delusion of Accountability in the Adult Education 

and Literacy System (AELS) of the United States. Retrieved June 21, 2006, from 
http://www.nald.ca/fulltext/sticht/may04/may21.htm 

The author discusses a report from the U.S. Department of Education (2003) and suggests it 
is propagating "delusional beliefs" because it presents information that is supposed to be 
used to hold programs accountable, but is based on arbitrary benchmarks and tests that are 
not comparable from state to state. The report, he notes, favourably compares such 
"objective" assessments with "subjective assessments" by teachers, while also dismissing the 
opinions of learners. Sticht argues that teachers and students are better equipped to decide 
whether programs are meeting their needs, and laments that current delusional beliefs about 
accountability preclude trusting them to make such decisions without outside interference. 
[PB] 


Asselin, M.; Early, M.; Filipenko, M. (2005). Accountability, Assessment, and ICT 
Literacies in the Canadian Context. Canadian Journal of Education, Volume 28, 
Number 4. Available at http://www.csse.ca/CJE/ 

This article explores the issues faced by Ministries of Education in their assessment policies 
as they try to account for the "new literacies" of information technology. While governments 
and industry exhort educators to expand the notion of literacy to include this new 
technology, the provinces are giving increasing importance to province-wide assessments 
that continue to exclude the new literacies. By analysing quantitative data and conducting 
interviews, the authors found that the provinces were still years away from including new 


23 


literacies in any meaningful way. They argue that traditional assessment programs cannot 
adequately measure these skills, and they call for a more balanced approach that recognizes 
the limitations of current public accountability measures while moving toward a broader 
assessment framework that includes alternative assessment techniques. [PB] 


Bianco, J.L. (2006). Processes of Policy Making and Theories of Public Policy: 

Relating Power, Policy and Professional Knowledge in Literacy Agendas 
Literacy Across the Currie uluMedia Focus, Volume 18, No. 1 
http://www.centreforliteracy.qc.ca/publications/lobianco.pdf 

The author argues that understanding policy is itself a form of literacy that literacy educators 
and researchers need to attend to if they are to have a chance of influencing the policies that 
affect their work. They tend to experience policy as an almost endless sequence of intrusions 
into their work - intrusions that notably include the imposition of ever more stringent ki nds 
of external accountability for the outcomes of literacy programs. The author maps out the 
ideological evolution affecting public policy on adult literacy that has led governments to 
reduce resources to the adult literacy sector while at the same time making increasingly 
stringent demands for “results”. To engage these policies in a democratic debate requires a 
critical understanding of the ideological presuppositions underlying them. [PB] 
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